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ABSTRACT 

This  project  aims  to  address  and  solve  key  sensing,  learning,  strategic  decision  and  resource  allocation  issues  in  the  design  of  a  cognitive 
tactical  radio  (CTR).  Such  a  tactical  radio  has  superior  ability  in  learning  its  spectrum  environment,  possibly  highly  dynamic,  and  can 
behave  intelligently  and  strategically  in  the  presence  of  multiple  other  completing  or  collaborating  tactical  radios.  Our  research  agenda 
consists  of  the  following  four  integral  tasks:  (Tl)  robust  spectrum  sensing  of  multiple  channels  using  adaptive  group  testing,  (T2)  multiuser 
learning  and  its  regret,  (T3)  convergence  of  multiuser  learning  to  pure  Nash  equilibrium  (  PNE)  in  spatial  congestion  games,  and  (T4) 
globally  optimal  spectrum  sharing  through  game-theoretic  and  mechanism  design-theoretic  frameworks.  We  have  successfully  completed 
these  tasks.  The  research  undertaken  by  the  project  can  have  significant  impact  both  theoretically  and  in  practice.  In  particular,  our 
contributions  in  the  areas  of  game  theory  and  online  learning  theory  are  quite  general  and  therefore  more  broadly  applicable  to  other  areas  of 
networking  as  well  as  decentralized  multi-agent  systems,  well  beyond  the  context  of  cognitive  radio  design. 
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(1 )  Statement  of  the  problem  studied 

This  project  aims  to  address  and  solve  key  sensing,  learning,  strategic  decision  and  resource  allocation  issues  in  the  design  of 
a  cognitive  tactical  radio  (CTR).  Such  a  tactical  radio  has  superior  ability  in  learning  its  spectrum  environment,  possibly  highly 
dynamic,  and  can  behave  intelligently  and  strategically  in  the  presence  of  multiple  other  completing  or  collaborating  tactical 
radios.  The  principal  goal  is  for  a  large  number  of  such  radios  to  coexist  efficiently  in  the  presence  of  either  hostile  or 
non-responsive  devices. 

This  project  is  of  significant  relevance  to  the  Army.  Wireless  spectrum  is  a  limited  resource  and  is  getting  ever  more  crowded. 
How  to  control  and  manage  spectrum  congestion  and  interference  thus  has  become  a  critical  issue  for  data  networking.  Recent 
advances  in  software  defined  ratio  (SDR)  and  cognitive  ratio  (CR)  have  opened  up  new  avenues  of  addressing  the  problem. 
However,  a  large  part  of  existing  literature  on  CR  focuses  on  a  primary-secondary  user  model  that  is  more  relevant  to 
civilian/commercial  applications  than  to  military  networks.  In  terms  of  theoretical  foundations,  while  the  research  community 
has  accumulated  a  large  volume  of  results  on  spectrum  sensing,  the  learning  and  reasoning  aspects  having  been  relatively 
lagging. 


(2)  Challenges 

We  note  that  the  challenges  facing  the  Army  in  the  context  of  cognitive  tactical  radio  design  are  unique:  (1)  the  spectrum 
environment  that  the  military  faces  is  highly  dynamic,  frequently  adversarial,  and  often  unknown,  due  to  the  existence  of  hostile 
devices,  and  (2)  the  type  of  wireless  devices  involved  are  often  highly  heterogeneous  and  are  used  for  drastically  different 
applications  with  different  load  and  quality  of  service  requirements  which  may  or  may  not  be  known  to  each  other  (i.e.,  they  are 
private  information).  Consider  for  instance  the  personal  area/body  sensor  networks  used  by  military  personnel.  Soldiers  are 
equipped  with  wireless  sensors  that  can,  monitor  their  health  status  and  provide  such  data  in  real  time  to  a  remote  command 
and  control  center,  or  monitor  the  immediate  environment  of  the  person  to  assist  real-time  decision-making.  This  is  in  addition 
to  other  wireless  voice/video/data  communication  devices  the  soldier  is  equipped  with.  In  such  scenarios,  many  of  these 
devices  will  likely  belong  to  different  networks,  each  serving  a  different  purpose  and  each  having  its  own  data  characteristics 
and  performance  requirements,  but  also  likely  having  to  share  the  same  spectrum.  Therefore  good  design  that  allows  efficient 
spectrum  sharing  and  coexistence  among  highly  heterogeneous  devices  and  networks  in  a  highly  dynamic  environment 
becomes  critical  for  military  networks,  and  this  is  what  the  project  intends  to  accomplish. 

(3)  Approach  taken  by  the  project 

Our  approach  consists  of  a  comprehensive  framework,  whereby  a  CTR’s  cognitive  ability  is  built  on  the  following  key 
components:  (1 )  information  acquisition  through  highly  efficient  and  robust  sensing  algorithms  that  exploit  sparse  and 
compressive  sampling  theory,  group  testing  theory  and  robust  algorithm  design  in  an  adversarial  setting;  (2)  learning  through 
state-of-the-art  online  algorithm  design;  and  (3)  strategic  decision  making  through  game-theoretic  and  mechanism 
design-theoretic  methods.  Accordingly,  our  research  agenda  consists  of  the  following  four  integral  tasks:  (T1)  robust  spectrum 
sensing  of  multiple  channels  using  adaptive  group  testing,  (T2)  multiuser  learning  and  its  regret,  (T3)  convergence  of  multiuser 
learning  to  pure  Nash  equilibrium  (PNE)  in  spatial  congestion  games,  and  (T4)  globally  optimal  spectrum  sharing  through 
game-theoretic  and  mechanism  design-theoretic  frameworks. 


(4)  Summary  of  the  most  important  results 

There  are  four  research  tasks  under  this  project  (T1-T4  as  listed  above).  We  have  successfully  completed  these  tasks.  Our 
most  important  results  are  summarized  in  the  following.  Firstly,  we  are  one  of  the  first  to  study  robust  spectrum  sensing  in  an 
adversarial  setting,  which  makes  our  results  relevant  in  a  hostile  environment  and  ensures  that  our  algorithms  can  function  in 
the  worst-case  scenarios.  Secondly,  we  developed  a  number  of  algorithms  for  decentralized  multi-user  online  learning  with 


order-optimal  regret  in  two  cases:  (1 )  when  the  optimal  channel  allocation  is  orthogonal,  i.e.,  no  more  than  one  user  on  a 
channel  (reference  2  below),  and  (2)  when  the  optimal  channel  allocation  may  involve  sharing  among  users  (reference  1, 6  and 
7  below).  To  the  best  of  our  knowledge,  this  second  case  has  not  been  studied  before.  Thirdly,  we  also  successfully 
developed  adaptive  learning  algorithms  that  can  achieve  order-optimal  strong  regret,  which  is  with  respect  to  the  best  dynamic 
policy  (see  reference  1 1  below).  Most  the  work  in  the  literature  (including  some  of  our  own)  focuses  on  weak  regret,  which  is 
with  respect  to  the  best  static  policy  (or  single-action  policy).  Our  result  is  therefore  much  stronger  in  terms  of  the  performance 
of  the  learning  algorithm  (at  an  expense  of  increased,  though  quantifiable,  computational  complexity). 

(5)  List  of  publications 

Book  chapter: 

1 .  C.  Tekin  and  M.  Liu,  “Performance  and  convergence  of  multiuser  online  learning  and  its  applications  in  dynamic  spectrum 
sharing,”  to  appear  in  the  book  Mechanism  and  Games  for  Dynamic  Spectrum  Allocation,  to  be  published  by  Cambridge 
University  Press. 

Journal  papers,  appeared  and  under  submission: 

2.  C.  Tekin  and  M.  Liu,  “Online  Learning  of  Rested  and  Restless  Bandits,”  in  IEEE  Transactions  on  Information  Theory,  vol. 
58,  no.  8,  pp  5588-5611,  August  2012. 

3.  Y.  Liu,  M.  Liu  and  J.  Deng,  “Is  diversity  gain  worth  the  pain:  performance  comparison  between  opportunistic  multi-channel 
MAC  and  single-channel  MAC,”  under  submission  to  the  IEEE  Journal  on  Selected  Areas  in  Communications  (JSAC). 

4.  C.  Tekin,  M.  Liu,  R.  Southwell,  J.  Huang,  and  S.  Ahmad,  “Spectrum  sharing  as  spatial  congestion  games,”  IEEE 
Transactions  on  Networking,  vol.  20,  no.  5,  pp.  1527-1540,  October  2012. 

5.  Y.  Gai,  B.  Krishnamachari  and  M.  Liu,  “Combinatorial  Network  Optimization:  A  Restless  Multi-Armed  Bandit  Approach”, 
submitted  to  IEEE  Transactions  on  Networking,  October  2012. 

6.  C.  Tekin  and  M.  Liu,  “Online  learning  in  decentralized  multiuser  resource  sharing,”  submitted  to  IEEE  Journal  of  Selected 
Topics  in  Signal  Processing  (JSTSP),  October  2012. 

Conference  publications: 

7.  C.  Tekin  and  M.  Liu,  “Online  learning  in  decentralized  multi-user  spectrum  access  with  synchronized  explorations,”  in  IEEE 
Military  Communication  Conference  (MILCOM),  October  2012,  Orlando,  FL. 

8.  Y.  Gai,  B.  Krishnamachari  and  M.  Liu,  “Online  learning  for  combinatorial  network  optimization  with  restless  Markovian 
rewards,”  IEEE  Conference  on  Sensor,  Mesh  and  Ad  Hoc  Communications  and  Networks  (SECON),  June  2012,  Seoul,  Korea. 
[Best  Paper  Runner-up] 

9.  C.  Tekin  and  M.  Liu,  “Approximately  optimal  adaptive  learning  in  opportunistic  spectrum  access”,  in  IEEE  Annual 
Conference  on  Computer  Communications  (INFOCOM),  March  2012. 

10.  Y.  Liu,  M.  Liu  and  J.  Deng,  “Is  diversity  gain  worth  the  pain:  performance  comparison  between  opportunistic  multi-channel 
MAC  and  single-channel  MAC,”  in  IEEE  Annual  Conference  on  Computer  Communications  (INFOCOM)  mini-conference, 
March  2012. 

11.  C.  Tekin  and  M.  Liu,  “Adaptive  learning  of  uncontrolled  restless  bandits  with  logarithmic  regret”,  in  Annual  Allerton 
Conference  on  Communication,  control  and  Computing  (Allerton),  pp.  1-8,  September  2011,  Allerton,  IL. 


(6)  Collaborations  and  Leveraged  Funding 

1.  We  are  collaborating  with  Prof.  Jianwei  Huang  at  the  Chinese  University  of  Hong  Kong,  on  research  issues  related  to  the 
convergence  and  performance  properties  of  congestion  games,  and  their  generalization  to  graphs.  Results  have  been  reported 
in  paper  4  listed  above  and  are  related  to  task  (T3). 

2.  We  are  collaborating  with  Prof.  Bhaskar  Krishnamachari  at  the  University  of  Southern  California,  on  the  use  of  our  online 
learning  algorithms  in  a  combinatorial  and  multiuser  setting.  This  result  has  been  reported  in  papers  5  and  8  listed  above,  and 
is  related  to  task  (T2). 

3.  We  are  collaborating  with  Prof.  Jing  Deng  at  the  University  of  North  Carolina,  Greensboro,  on  the  performance  comparison 
study  of  multi-channel  opportunistic  MAC  vs.  single-channel  MAC.  Preliminary  result  is  included  in  papers  3  and  10  listed 
above  and  is  related  to  task  (T4). 

4.  We  have  on-going  collaboration  with  Prof.  Romesh  Saigal  at  the  Industrial  and  Operations  Engineering  Department  at  the 
University  of  Michigan  on  refining  a  stochastic  differential  equation  (SDE)  based  spectrum  model  we  developed  earlier.  This 
model  is  going  to  be  used  for  the  evaluation  of  algorithms  developed  under  this  project. 

(7)  Conclusion 

We  have  successfully  completed  the  proposed  research  tasks,  and  made  significant  progress  toward  our  ultimate  goal  of 
building  a  versatile  and  agile  cognitive  radio.  The  research  undertaken  in  this  project  is  of  significant  relevance  to  the  Army, 
and  makes  significant  contribution  to  both  the  theory  and  practice  of  cognitive  radios. 
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